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LONG  TERM  GOALS 

The  goal  of  this  project  is  to  develop  methods  for  automated  analysis  of  zooplankton  size  and  taxonomic 
composition  using  images  from  the  Video  Plankton  Recorder  (VPR).  Ultimately,  the  system  will  sort 
plankton  to  species  level  and  measure  body  size  in  real  time  at  sea.  The  VPR  provides  sharply  focussed  video 
images  of  plankton  and  seston  in  the  size  range  100  microns  to  5  cm  and  has  an  on-board  CTD,  fluorometer, 
flowmeter,  and  transmissometer.  Currently  the  VPR  is  towed  at  0.5 -5.0  m/s  and  has  been  deployed  in  shelf  and 
oceanic  waters,  including  extensive  survey  work  on  Georges  Bank.  The  VPR  has  also  been  deployed  on  the  ROV 
JASON  for  individual  particle  tracking,  and  a  moored  profiling  system  is  under  development,  which  will  include  on¬ 
board  image  processing  and  satellite  telemetry  of  processed  data. 

OBJECTIVES 

Our  four  specific  objectives  were  to:  1)  use  our  existing  hardware  to  develop  methods  for  real-time  detection  (60 
fields  per  second)  of  in-focus  organisms  in  the  video  and  for  storing  the  images  to  disk;  2)  develop  pattern- 
recognition  software  for  classification  of  organisms  into  major  taxonomic  groupings  (copepods,  chaetognaths, 
doliolids,  etc.);  3)  develop  specifications  for  transferring  the  software  routines  to  hardware  to  enable  real-time 
sorting  of  zooplankton  according  to  major  taxa;  and  4)  to  develop  a  data-analysis  and  display  system  for  real-time 
visualization  of  plankton  distributions.  Development  of  the  complete  image  processing/data  analysis  package 
enables  incorporation  into  the  VPR  system  for  real-time  identification  of  planktonic  taxa  and  visualization  of  their 
distributions  while  at  sea. 

APPROACH 

Our  approach  for  the  first  objective  was  to  develop  software  for  accurate  focus  detection  and  then  transfer  it  to  our 
IT-151  image  processor.  The  approach  for  the  second  objective  was  to  develop  software  for  identification  and 
sorting  organisms  to  major  taxa.  The  third  objective  involved  establishing  specifications  for  hardware  that  operates 
the  identification  and  sorting  software  in  real-time.  The  fourth  objective  was  to  develop  a  data  analysis  system  for 
viewing  the  spatial  patterns  at  sea  in  real  time. 

WORK  COMPLETED 

We  have  achieved  all  objectives  of  this  project  and  have  used  the  prototype  system  at  sea  to  identify  and  visualize 
planktonic  taxa  in  real  time  (Davis  et  al,  submitted).  This  project  has  involved  development  of  a  real-time  focus 
detection  system,  development  of  algorithms  for  feature  extraction  and  identification  of  planktonic  taxa,  and 
development  of  data  visualization  software  for  display  of  plankton  and  physical  data. 
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RESULTS 


Objective  1:  We  have  developed  accurate  focus-detection  algorithms  and  implemented  them  in  hardware.  Initially 
this  was  done  using  an  Imaging  Technology  Inc.  Model  151  (ITI-151)  real-time  image  processor  interfaced  to  a  Sun 
SPARCstation  20,  but  we  have  upgraded  die  system  to  an  ITI  Model  150-40  running  on  a  Pentium  PC  to  take 
advantage  of  the  rapid  I/O  of  the  PCI  bus.  The  old  system  missed  in- focus  objects  and  would  skip  frames  due  to  an 
I/O  bottle  neck  in  the  Sparc  S-Bus.  The  new  PCI  based  system  is  much  faster  and  is  capable  of  extracting  all  in¬ 
focus  images.  A  video  source  (live  camera  or  tape  player)  sends  a  video  signal  and  time  code  to  the  ITI  150-40  and 
to  a  separate  time-code  reader  connected  to  the  PC  serial  port.  Algorithms  were  developed  for  reading  field- 
accurate  (60-Hz)  time  code  into  the  PC  and  for  synchronizing  the  ITI  150-40  pipeline  processor  for  real-time  region- 
of-interest  (ROI)  selection.  In- focus  ROIs  then  are  stored  on  disk  as  TIF  files. 

Our  capability  for  real-time  focus  detection  allows  for  rapid  automated  culling  of  blank  and  out-of-focus  images, 
reducing  manual  video  processing  by  a  factor  of  more  than  50.  We  have  used  the  focus-detection  system  coupled  to 
a  point-and-click  user  interface  for  rapid  analysis  of  zooplankton  distributions  at  sea  (Davis  et  al.,  1996).  We  used 
this  method  on  our  Georges  Bank  GLOBEC  cruises  to  observe  2-  and  3-D  plankton  distributions  (Gallager  et  al., 
1996;  Benfield  et  al.,  1996;  Norrbin  et  al.,  1996;  Benfield  et  al.,  submitted;  Ashjian  et  al.,  in  draft;  Davis  et  al.,  in 
prep;  Gallager  et  al.,  in  prep). 

Objective  2:  In  addition  to  focus  detection,  we  also  have  developed  feature-extraction  and  pattern- analysis 
algorithms  for  classification  according  to  taxonomy  and  size.  Our  new  algorithms  have  classification  accuracy  at 
about  90%  for  several  plankton  taxa.  Our  research  efforts  have  focused  on:  1)  gray-scale  morphology  and 
granulometric  feature  extraction;  2)  textural  feature  extraction;  and  3)  feature  orthogonality  and  optimal 
discrimination. 

1)  For  Binarization,  Dr.  Luc  Vincent  has  developed  a  new  approach  to  using  a  global-connectivity  histogram  of  the 
image;  histogram  plateaus  indicate  which  threshold  values  are  most  stable.  Granulometric  algorithms  were 
improved,  especially  with  respect  to  scale,  filtering,  and  normalization;  classification  results  indicate  good 
discrimination  for  plankton  images.  A  novel  "geodesic  radial  function,"  the  ordered  set  of  values  of  the  propagation 
function  along  the  boundary  of  the  binarized  plankton  mask  is  an  important  feature.  The  function  captures  the  size 
and  elongation  of  the  organism,  its  extremities  and  their  sharpness,  and  overall  smoothness. 

2)  Postdoctoral  Investigator  Xiaoou  Tang  has  developed  a  recursive  segmentation  scheme  inspired  by  unsupervised 
clustering;  the  technique  offers  speed,  simplicity,  and  performance  comparable  with  the  global-connectivity 
histogram.  Next,  a  set  of  geometrical  features  (e.g.,  area,  perimeter,  shape  factor,  rotation-  and  scale-invariant 
moments)  are  computed  from  the  binary  segmented  image.  Other  textural  features  (e.g.,  run  length,  co-occurrence, 
gray-level  difference,  wavelet,  power  spectrum,  fractal  dimension)  are  also  extracted  from  the  original  gray-scale 
images.  These  and  other  feature  vectors  were  developed,  including  the  geodesic  radial  function  and  a  boundary 
shape  descriptor,  and  incorporated  into  a  robust  feature  set. 

3)  With  very  large  feature  vectors  much  information  is  redundant,  demanding  great  computational  power  and  also 
showing  poor  results.  An  important  step  is  the  decorrelation  and  feature  selection.  Tang  developed  a  new  dominant 
principal  component  analysis  approach  on  the  combined  large  feature  vectors  formed  by  various  features  described 
above.  Then  the  selected  dominate  features  are  sorted  according  to  their  discrimination  power  by  the  Bhattacharyya 
distance  measure.  Dramatic  improvement  of  the  classification  accuracy  is  achieved  by  only  using  a  simple 
statistical  classifier. 

Objective  3:  Once  the  training  sets  and  related  software  were  established,  the  focus,  sizing,  and  identification 
algorithms  allow  completely  automated  analysis  of  VPR  images.  We  have  used  the  feature  extraction  and 
classification  algorithms  on  our  workstations  to  achieve  real-time  processing  and  classification  of  the  VPR  images 
as  they  are  acquired  at  sea. 
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IMPACT 


The  new  sampline  capability  developed  in  this  project  will  provide  new  insights  into  the  processes  controlling 
plankton  distributions  and  thus  sound  and  light  transmission  in  the  sea.  The  ability  to  visualize  the  distribution  of 
planktonic  taxa  (and  size  structure)  at  sea  in  real  time  allow  for  a  better  understands  of  pelagic  biology.  No 
longer  will  we  be  probing  in  the  dark,  only  able  to  analyze  the  point  source  data  once  samples  have  been  processed 
in  the  laboratory  months  or  years  later.  The  dynamical  nature  of  the  pelagic  environment  necessitates  the  use  of  tools 
which  can  provide  rapid  visualizations  of  plankton  distributions  together  with  physical  properties  of  the  water.  The 
prototype  system  we  have  developed  provides  this  capability. 

TRANSITIONS 

The  real-time  sampling  system  can  be  used  in  any  studies  in  which  it  is  desirable  to  measure  plankton  abundance 
including  sound  and  light  scattering  studies  and  ecological  studies  of  environmental  quality  and  fisheries  ecology. 

RELATED  PROJECTS 

The  video  processing  procedures  we  have  developed  are  closely  coupled  to  our  NSF/NOAA  funded  research  on 
Georges  Bank  (GLOBEC)  and  our  NSF-funded  development  of  a  vertically  profiling  VPR. 
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